Position estimating apparatus, position estimating method, and terminal apparatus

ABSTRACT

According to one embodiment, an electronic apparatus includes receiver circuitry and controller circuitry. The receiver circuitry is configured to receive a radio signal from a target apparatus. The controller circuitry is configured to estimate first position information of the target apparatus based on a captured image; and estimate second position information of the target apparatus based on both the first position information and information of the radio signal.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of U.S. patent application Ser. No.15/902,253, filed on Feb. 22, 2018, which is based upon and claims thebenefit of priority from Japanese Patent Application No. 2017-230795,filed on Nov. 30, 2017, the entire contents of all of the aboveapplications are incorporated herein by reference.

FIELD

Embodiments described herein relate to a position estimating apparatus,position estimating method, and terminal apparatus.

BACKGROUND

Technologies for estimating the position of an object using a cameraimage or radio communication have been extensively studied. The methodusing the camera can perform distance measurement to the object withhigh accuracy, and with a plurality of cameras, it is possible toestimate two-dimensional and three-dimensional position of the object.On the other hand, there is a disadvantage of having difficulty inmeasuring an object far from the camera, and thus, having difficulty ingrasping the object even by image processing depending on the situation.With a method using radio communication, it is possible to estimate theposition of the object over a wide area and detect an identifier (ID) ofthe object from the communication content to identify the object. On theother hand, an influence of attenuation, reflection, diffraction in theradio propagation path, an influence of radio circuit distortion, or thelike, might increase an error, leading to a disadvantage of havingdifficulty in performing highly accurate position estimation.

In order to solve this problem, a position estimation technology usingboth radio communication and camera image has been developed. Forexample, a mobile body equipped with a global positioning system (GPS)receiver and a camera detects its self-position with high accuracy. Theposition is estimated by the GPS at a place where GPS reception isavailable, and the amount of movement is grasped by a camera image toestimate the self-position at a place where GPS reception isunavailable. However, conventional techniques use position estimationresults by radio (GPS) in the place where the radio (GPS) can bereceived, making it difficult to improve estimation errors due to radio(e.g., errors due to propagation path, noise, synchronization errors).Moreover, this technique assumes GPS alone as a radio, making itdifficult to be applied in areas where GPS is not available.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a configuration of a positionestimating apparatus according to an embodiment;

FIG. 2 is a block diagram illustrating functions of a position estimatoraccording to an embodiment;

FIG. 3 is a flowchart illustrating operation of a position estimatingapparatus according to an embodiment;

FIG. 4 is a diagram illustrating an installation example of a positionestimating apparatus according to an embodiment;

FIG. 5 is a block diagram illustrating another example of a positionestimator according to an embodiment;

FIG. 6 is a flowchart illustrating another example of operation of aposition estimating apparatus according to an embodiment;

FIG. 7 is a block diagram illustrating still another example of aposition estimator according to an embodiment;

FIG. 8 is a diagram illustrating a configuration of a positionestimating apparatus according to an embodiment; and

FIG. 9 is a block diagram illustrating functions of a target terminalaccording to an embodiment.

DETAILED DESCRIPTION

According to one embodiment, an electronic apparatus includes receivercircuitry and controller circuitry. The receiver circuitry is configuredto receive a radio signal from a target apparatus. The controllercircuitry is configured to estimate first position information of thetarget apparatus based on a captured image; and estimate second positioninformation of the target apparatus based on both the first positioninformation and information of the radio signal.

Hereinafter, embodiments will be described in detail with reference tothe drawings. In the explanation, a person or thing to be an object ofposition estimation is called a “target”. For example, it is assumedthat the target carries (mounts) a target terminal such as a smartphone,a tablet or a beacon, capable of transmitting and receiving radio waves.Alternatively, the target terminal itself may be a target.

First Embodiment

A position estimating apparatus according to the present embodimentintends to complementarily enhance the accuracy of the positionestimation of the target using both the position estimation by thecamera imaging and the position estimation by radio.

FIG. 1 is a diagram illustrating a configuration example of a positionestimating apparatus 1 according to the present embodiment. The positionestimating apparatus 1 includes a position estimator 10, an imager 20,and a communicator 30, and estimates the position of a target includinga target terminal 40.

While this is an exemplary case including one imager 20 and fourcommunicators 30, namely, communicators 30A to 30D, the configuration isnot limited to this example, and any number is allowable as long asposition information is appropriately obtained. For example, two or moreimagers 20 may be provided, and one, two, three, or five or morecommunicators 30 may be provided. Moreover, the number of the targetterminals 40 is not limited to one, and a plurality of target terminals40 may be present, and the position estimation of the plurality oftarget terminals 40 may be performed at the same timing in parallel orat mutually different timings.

While the position estimator 10, the imager 20, and the communicator 30are configured separately, the configuration is not limited to thisexample, and may be configured as an integrated unit. Alternatively, aportion of the functions of the position estimator 10 may be mounted onthe imager 20 or the communicator 30, and the remaining functions may bemounted on an external device. Furthermore, there is no need to providethe imager 20 in the position estimating apparatus 1, and a camera orthe like installed separately from the position estimating apparatus 1may substitute the imager 20.

For example, the plurality of imagers 20 and the plurality ofcommunicators 30 may be installed to estimate the position of a movingvehicle or work vehicle in a factory, track a flow line of a visitor ina shopping mall, or monitor patient behavior within a hospital.

An example of the imager 20 is a camera to capture an image within apredetermined region and output the captured image to the positionestimator 10. The imaging may be performed at predetermined timings ormay be performed in a case where a target is detected to be presentwithin a predetermined region by a separate detector. Alternatively, theimaging may include imaging of a moving image and outputting the imageto the position estimator 10.

The communicator 30 includes at least a receiver to receive a radio, andreceives the radio (e.g., radio waves) information from the targetterminal 40. The radio information may be anything available forestimating the position of the target terminal 40, and may be receivedpower, reception timing, reception direction, for example.Alternatively, in addition to these pieces of information, there may beinformation indicating a decoding result of a radio signal, an errorvector magnitude (EVM) of the radio signal, and a frequency spectrum, orthe like. The received information is output to the position estimator10. In addition to this, the communicator 30 may include a transmitterto transmit the information estimated by the position estimator 10 tothe target terminal 40.

FIG. 2 is a block diagram illustrating an example of functions of theposition estimator 10. The position estimator 10 includes a firstestimator 12 and a second estimator 14. Although not illustrated, theremay be a position estimation controller to control these positionestimators. Moreover, there may be a controller to control the functionsof the components of the position estimating apparatus 1. That is, thecontroller (not illustrated) may perform various types of controls ofthe position estimating apparatus 1 described below. This controller maybe mounted as a control circuit on a processor configured with a digitalor analog circuit.

The first estimator 12 estimates first position information on the basisof the information imaged by the imager 20. The first positioninformation is position information obtained from the image captured bythe imager 20 and is estimated by analyzing the image. Examples of thefirst position information include a distance to the target,orientation, the number of targets, the coordinates of the target.

The second estimator 14 estimates second position information on thebasis of the estimated first position information and the communicationfrom the target received by the communicator 30. In this manner, thesecond position information is estimated by analyzing the image capturedby the imager 20 and the data received from the communicator 30.

The second estimator 14 includes a learner 140 and a calculator 142. Thelearner 140 performs learning to enhance accuracy of position estimationfrom a state of radio waves received by the communicator 30 on the basisof the first position information. The learner 140 calculates, forexample, parameters for estimating a position from the received radiowaves, by learning. As another example, it is allowable to generate amodel such as a neural network.

Using the parameter or model learned by the learner 140, the calculator142 estimates and outputs the second position information from the firstposition information and the state of the radio waves received by thecommunicator 30 by calculating the second position information. Theinformation may be output via standard output of the position estimator10 or to an internal memory, or may be transmitted as a radio signalfrom the communicator 30 and a position estimation result may benotified to the target terminal.

FIG. 3 is a flowchart illustrating operation of the position estimatingapparatus 1. Hereinafter, operation of the position estimating apparatus1 will be specifically described using this flowchart.

First, the position estimating apparatus 1 obtains data related to thetarget terminal 40 (S100). The data related to the target terminal 40 isdata of an image of the target terminal 40 captured by the imager 20 orthe state of the radio waves transmitted from the target terminal 40,received by the communicator 30.

Next, the first estimator 12 estimates the first position information(S102). For example, the first estimator 12 estimates the position ofthe target terminal 40 on the basis of information such as the positionand the size of the target terminal 40 included in the image captured bythe imager 20, within the image. This algorithm is not particularlylimited, as long as it is an algorithm capable of appropriatelycalculating the position from the image. In a case where the pluralityof imagers 20 is provided, it is also possible to enhance the accuracyby using the images captured by the plurality of imagers 20.

Next, the second estimator 14 determines whether the first positioninformation has been obtained as appropriate position information(S104). For example, in a case where an appropriate result such as adistance to the target terminal 40 and coordinates has been output asthe first position information, it is determined that highly accurateposition estimation is possible from the image captured by the imager20.

As another example, when the first estimator 12 estimates the firstposition information, the determination of S104 may be performed byexecuting correlation calculation with the image of the target terminal40, calculating a sharpness value of the target terminal 40 anddetermining whether the sharpness value exceeds a threshold. The objectof imaging by the imager 20 is not limited to the target terminal 40,and may be an image of a person carrying the target terminal 40 or animage of anything on which the target terminal 40 is present.

In a case where it is determined that appropriate position informationhas been obtained as the first position information (S104: YES),parameters (hereinafter referred to as radio parameters) used for radioposition estimation processing is learned (S106) on the basis of thefirst position information. The radio parameters include a time offsetof radio communication, a coefficient of a propagation path model, or acoefficient used in a signal processing algorithm, and the accuracy ofposition estimation is determined by these parameters. Details onlearning of radio parameters will be described below.

Next, the second position information is estimated (S108) on the basisof the first position information and the state of the radio wavesreceived by the communicator 30. In a case where it is determined inS104 that sufficient accuracy has been obtained as the first positioninformation, the first position information may be output as it is asthe second position information, or may be corrected or arithmeticallyprocessed using the radio reception information to be used as the secondposition information. The state of the received radio waves may beadditionally used in this correction and arithmetic processing. Thecorrection using the radio reception information may be any correctionas long as the first position information estimated from the imaginginformation is corrected on the basis of provisional positioninformation estimated from the radio reception information. Exemplarycorrection may be correction to weight-average the first positioninformation estimated from the imaging information and the provisionalposition information estimated from the radio reception information.

In contrast, in a case where it is determined that appropriate positioninformation has not been obtained as the first position information(S104: NO), it is determined that appropriate detection of the targetterminal 40 failed on the imager 20, and radio-based position estimationis performed in the second estimator 14 (S110). The second estimator 14performs position estimation using the radio parameter already learnedby the learner 140 or the radio parameter held in advance, and thecalculator 142 calculates and estimates the second position information.

In this manner, in a case where the target terminal 40 is detectable bythe imager 20, the radio parameter learning is performed whileimage-based position estimation is performed. In a case where the targetterminal 40 is not detectable by the imager 20, the learned radioparameter or the radio parameter previously held is used to performposition estimation on the basis of the radio information received bythe communicator 30. This makes it possible to appropriately learn theradio parameters to enhance the accuracy of the position estimationprocessing using the radio.

FIG. 4 is a diagram illustrating another example of installation of theposition estimating apparatus 1. Unlike FIG. 1, this figure includes anobstacle and a position where imaging of the target or the targetterminal 40 by the imager 20 would be difficult.

A case where there is a shield 50 to shield the imager 20 within aregion where position estimation is performed will be examined. In thiscase, in a situation that the target terminal 40 retreats to thebackside of the shield 50 with respect to the imager 20, imaging of thetarget terminal 40 might be hindered, making it difficult to estimatethe distance on the basis of the imaged information.

In a case where the target terminal 40 is moving along a route 400within the range visible from the imager 20, the imager 20 can recognizethe target terminal 40, and thus, the imaging information of the imager20 can be used to perform the first position estimation. At the timingwhen it is possible to perform the first position estimation, thelearner 140 performs learning for the second position estimation.

In another case where the target terminal 40 is moving along a route 402outside the visible range of the imager 20, the calculator 142 of thesecond estimator 14 calculates and estimates the second positioninformation on the basis of the radio reception signal received by thecommunicator 30 and using the radio parameter learned above.

In this manner, the learner 140 learns the radio parameters on the basisof the estimation result of the first position information within therange that can be imaged by the imager 20, making it possible to enhancethe accuracy of position estimation using radio even in a case where thetarget terminal 40 is present in a range that cannot be imaged by theimager 20 because of the shield 50. This is because highly accurateposition estimation results by the imager 20 such as a camera areavailable as teacher data for learning the radio parameters.

Next, a specific example of the radio position estimation method andradio parameters will be described. Position estimation techniques viaradio include a method of using an arrival time of the radio signals, amethod of using a received power, a method of using an arrivaldirection.

In the use of the arrival time, an arrival time t_(i) of the radiosignal from the target terminal 40 in the ith communicator 30 of aplurality of communicators 30 (assumption) is given by the followingexpression.

$\begin{matrix}{t_{i} = {\frac{d_{i}}{c} + {\Delta\; T_{i}}}} & (1)\end{matrix}$where c is a speed of light, d_(i) is a distance between the ithcommunicator 30 and the target terminal 40, and ΔT_(i) is a time offset.This Formula (1) can be transformed into the following expression.

$\begin{matrix}\begin{matrix}{d_{i} = {( {t_{i} - {\Delta T_{i}}} ) \cdot c}} \\{\cong {t_{i} \cdot c}}\end{matrix} & (2)\end{matrix}$That is, when the time offset ΔT_(i) is sufficiently small, the distanced_(i) between each of the communicators 30 and the target terminal 40can be estimated by observing the arrival time t_(i) of the radiosignal.

At this time, by estimating the distance with at least threecommunicators 30, it is possible to calculate and estimate the positionof the target terminal 40 according to the principle of trilateration.

The time offset ΔT_(i) in Formula (1) can be a cause of estimationerrors. The time offset is an error caused by clock deviation betweenthe target terminal 40 and each of the communicators 30, reflection onthe propagation path, radio circuit distortion, or the like. When theinfluence of this time offset is large, Formula (2) would not beestablished, leading to deterioration of position estimation accuracy.

In the present embodiment, as an example, the time offset is defined asa radio parameter to perform learning using the estimation result of thefirst position information by the imager 20. In a case where the targetterminal 40 is present within an region that can be imaged by the imager20 as the route 400, highly accurate camera-based position estimation ispossible, in which an estimation result of the first positioninformation is assumed as [x0, y0]. Hereinafter, for the sake ofsimplicity, description will be given in a two-dimensional coordinatesystem, while the similar operation will apply to the three-dimensionalsystem.

When the coordinates of the ith communicator 30 are [xi, yi], thedistance from the target terminal 40 to the ith radio station can beapproximated as follows.d _(i)≅√{square root over ((x ₀ −x _(i))²+(y ₀ −y _(i))²)}  (3)From Formulas (1) and (3), the time offset ΔT_(i) to be the cause of anerror can be calculated as follows.

$\begin{matrix}{{\Delta\; T_{i}} \cong {t_{i}\frac{\sqrt{( {x_{0} - x_{i}} )^{2} + ( {y_{0} - y_{i}} )^{2}}}{c}}} & (4)\end{matrix}$

Every time the coordinates [x0, y0] estimated as the first positioninformation and reception time information t_(i) via the communicator 30are obtained, the value of the time offset in each of the communicators30 is calculated on the basis of Formula (4). After the time offset isobtained, it is possible to accurately calculate the distance to thetarget terminal 40 expressed by Formula (2), making it possible toenhance the accuracy of position estimation calculated on the basis ofthe distance.

In a case where the imager 20 can detect the target terminal 40,estimation of the first position information using the imaginginformation by the imager 20 is executed while the time offset as theradio parameter is estimated. In a case where the target terminal 40 isundetectable by the imager 20, the value of the estimated time offset sofar would be used to estimate the second position information based onthe arrival time of the radio, making it possible to achieve highlyaccurate position estimation.

To achieve an effect of enhanced accuracy of estimation, there is a needto have a same time offset in execution of camera-based positionestimation and radio-based position estimation. The time offset is notalways the same value because of the presence of fixed componentsattributed to the hardware of the target terminal 40 and thecommunicator 30 and variation components attributed to the propagationpath. Still, it is at least possible to reduce the influence of fixedcomponents.

Moreover, in the present embodiment, since the time offset iscontinuously calculated during execution of the camera-based positionestimation, it is possible to learn the time variation and statisticalproperties of the time offset. Therefore, in execution of theradio-based position estimation, the influence of time offset can bereduced using the learned statistical properties. The statisticalproperties may include an average value, a variance value, a probabilitydensity function of a radio parameter, for example.

With this technique, it is possible to reduce the influence of the timeoffset without calibrating the deviation of the clock between thecommunicators 30.

While the technique based on arrival time has been described as anexemplary position estimation technique by radio, a technique based onreceived power may also be applied as described above. The receivedpower of the radio signal can be defined in free space as follows.R _(i) =P _(tx)−20·log₁₀(d _(i))  (5)where R_(i) is a received power at the ith communicator 30, P_(tx) is atransmission power at the target terminal 40, and d_(i) is a propagationdistance. By transforming Formula (5), the following propagationdistance can be obtained.

$\begin{matrix}{d_{i} = {10^{\bigwedge}\{ \frac{( {P_{tx} - R_{i}} )}{20} \}}} & (6)\end{matrix}$

The actual position estimation environment is not necessarily a freespace, and thus, Formula (6) might not be established according to theinfluence of an obstacle, or the like, in many cases. Therefore, it isalso allowable to introduce a variation parameter a at the time ofposition estimation to define the distance d_(i) on the basis of thefollowing form.

$\begin{matrix}{d_{i} = {10^{\bigwedge}\{ \frac{( {P_{tx} - R_{i}} )}{\alpha} \}}} & (7)\end{matrix}$This variation parameter a is a parameter determined by the propagationenvironment. Application of an accurate parameter setting value would beable to enhance the estimation accuracy of the propagation distance toachieve accurate position estimation.

In a case of executing position estimation using the received power ofradio signals, a in Formula (7) is learned as a radio parameter. In acase where the target terminal 40 is detectable from the imager 20, itis possible to obtain the distance d_(i) on the basis of Formula (3),and thus, possible to use the known transmission power P_(tx) and theobserved value R_(i) of the received power to calculate the radioparameter a by Formula (7).

The radio parameter a is calculated and statistical properties thereofare learned every time the first estimator 12 estimates the firstposition information on the basis of the information obtained from theimager 20. In a case where the target terminal 40 is undetectable by theimager 20, the value of a learned so far or the learned statisticalproperties would be used to estimate the second position information onthe radio power basis, making it possible to achieve highly accurateposition estimation. In a case where the transmission power P_(tx) inthe target terminal 40 is unknown, P_(tx) may be used as a radioparameter in addition to a.

As still another example of the radio parameter, a weighting coefficientc_(i) representing the reliability of each of the communicators 30 maybe defined as a parameter. When observation data observed by the ithcommunicator 30 is defined as o_(i), the distance d_(i) between the ithcommunicator 30 and the target terminal 40 is expressed as follows.

$\begin{matrix}{d_{i} = {{f( o_{i} )} + \frac{1}{c_{i}}}} & (8)\end{matrix}$where f (⋅) is a function that converts the observation data to thedistance. When the observation data is the propagation time, this casecorresponds to Formula (2), and when the observation data is receivedpower, this corresponds to Formula (7). The portion 1/c_(i) in thesecond term corresponds to an observation error, and the greater theweighting coefficient c_(i) representing reliability, the smaller theerror.

As an example, this weighting coefficient c_(i) may be learned as aradio parameter. The learning method is similar to the case of otherparameters. Learning this weighting coefficient c_(i) makes it possibleto perform processing such as position estimation processing byweighting an observed value in each of the communicators 30 or positionestimation using the communicator 30 with less observation error,leading to enhancement of the estimation accuracy of the second positioninformation.

As still another example, a parameter related to the position estimationalgorithm or the filtering algorithm may be set as the radio parameter.The position estimation algorithm is an algorithm related to signalprocessing of calculating actual positioning coordinates fromobservation data in the communicator 30, and includes various techniquessuch as a triangulation method, a maximum likelihood estimation method,and a least squares method. The filtering algorithm is an algorithmrelated to signal processing of smoothing observation data orpositioning coordinates, and includes various techniques such as aKalman filter and a particle filter.

Furthermore, these position estimation and filtering algorithms may havetheir own internal parameters in some cases. That is, there are caseswhere a plurality of algorithms and parameter setting candidates areconceivable. In such a case, the present embodiment can also be appliedas a means for selecting an optimum candidate.

For example, the following is a case where there are the followingcandidates as combinations of algorithms and parameters.

Candidate 1: (Algorithm 1, parameter A)

Candidate 2: (Algorithm 2, Parameter B)

Candidate 3: (Algorithm 3, parameter C)

Candidate 4: (Algorithm 4, Parameter D)

In this case, the learner 140 may be configured to learn which candidateis most probable. Specifically, at the timing when the target terminal40 is in a region where the target terminal 40 can be imaged by theimager 20, estimation processing of the second position information bythe combination of the candidate algorithms and parameters is performedin parallel with the estimation processing of the first positioninformation based on the imaging information obtained by the imager 20.The degree of similarity in the estimation result by each of thecandidates with respect to the position estimation result by the imager20 is learned. In this learning, for example, a candidate having thehighest correlation with the first position information is employed forradio estimation processing.

The position estimation technique by radio include various observationmeans and algorithms other than those described above. In any case, thepresent embodiment is applicable with some radio parameter settings. Inthe case where highly accurate position detection is possible via acamera, by learning the value or statistical properties of the radioparameter and reflecting the learning result, it is possible to performhighly accurate position estimation processing in performing radio-basedposition estimation in a region where imaging by the camera is notavailable.

While the present embodiment illustrates a case where the camera is usedas a means for enabling highly accurate position estimation and theradio is used as a low-accuracy position estimating means, another meansmay be used. For example, it is allowable to have a configuration to usea positioning satellite with very high accuracy and less error insteadof the camera.

As described above, according to the present embodiment, positionestimation using the camera (imager 20) and position estimation usingradio (communication by the communicator 30) are complementarily used toenhance position estimation accuracy. That is, in a case where a targetis present in a range that can be imaged by the camera, highly accurateposition estimation based on the imaged information is performed whileparameters for position estimation using radio are learned. In a casewhere there is no target within the range that can be imaged by thecamera, position estimation using radio is performed using the learnedparameters, making it possible to perform highly accurate positionestimation even when the position estimation using the camera is notavailable.

Note that in a case where the environment in which the imager 20 and thecommunicator 30 are installed does not change so much, it is alsopossible to hold the radio parameters in advance as described above. Inthis case, for example, the parameters may be calculated and set on thebasis of installation situation of the imager 20 and the communicator 30or the like. As described above, in a case where the radio parametersare set in advance, there is no need to provide the learner 140 in thesecond estimator 14.

Second Embodiment

In addition to the above-described embodiment, the present embodiment isintended to obtain the reliability of the estimation result of the firstposition information obtained from the imaging information captured bythe imager 20 and to change the estimation method of the second positioninformation on the basis of the reliability.

FIG. 5 is a block diagram illustrating functions of the positionestimator 10 according to the present embodiment. The position estimator10 further includes a reliability acquirer 16 configured to receive theimaging information from the imager 20 and the first positioninformation from the first estimator and to obtain the reliability onthe basis of the imaging information to output the obtained reliabilityto the second estimator 14.

The reliability acquirer 16 obtains reliability of the first positioninformation on the basis of the imaging information received from theimager 20. Examples of means for judging the reliability include thepresence or absence of the target terminal 40 included in the imaginginformation, the size of the region occupied by the target terminal 40within the imaging information (the number of pixels within the image),the brightness of the imaging environment, and sharpness. As describedabove, while the position estimation by the imager 20 can achieve highlyaccurate estimation results, sufficient accuracy may not be obtaineddepending on the imaging environment.

For example, there is a possibility that sufficient accuracy cannot beobtained in an environment where the distance to the target terminal 40is long, the imaging region is dark, or the like. The reliabilityacquirer 16 calculates and obtains the reliability of the imaginginformation, thereby notifying the second estimator 14 as to whether theestimation of the first position information is performed withsufficiently high accuracy.

The second estimator 14 learns the radio parameters and estimates thesecond position information using the input first position information,the reliability thereof, and the radio reception information from thecommunicator 30.

FIG. 6 is a flowchart illustrating operation of the position estimatingapparatus 1 according to the present embodiment. In the figure, the samereference numerals as those in FIG. 3 denote the same processing.

After estimating the first position information by the first estimator12 (S102), the reliability acquirer 16 obtains reliability (S203) on thebasis of the imaging information transmitted from the imager 20 and thefirst position information estimated by the first estimator 12. Asdescribed above, the reliability is obtained from imaging information(image) captured by the imager 20.

For example, the number of pixels occupied by the target terminal 40 inthe image is obtained as the reliability. Other examples includestatistics of gray scale values of all the pixels in the image,statistics obtained from saturation and brightness of all pixels in theimage, values obtained from sharpness of a portion or whole of theimage. In the case of using sharpness, for example, a differentialfilter such as a Laplacian filter is applied to the image, edgedetection is performed by a Canny filter or the like, statistics of thepixel value after filtering are calculated to obtain the reliability.The statistics means at least one value among total, average, variance,or the like.

Next, the second estimator 14 determines whether the reliabilityobtained by the reliability acquirer 16 is a predetermined threshold ormore (S204). The predetermined threshold may be a preset value or may beautomatically determined by the second estimator 14 in operatinglong-term camera-based or radio-based position estimation. Note that“the threshold or more” may be read as a value larger than thethreshold.

When the reliability is the threshold or more, the operation from S106is performed, and if not so, the operation from S110 is performed.

As described above, according to the present embodiment, thecamera-based and the radio-based estimation can be performed to enhancethe accuracy of position estimation similarly to the above-describedembodiment. Furthermore, on the basis of the reliability of thecamera-based position estimation, it is determined whether to learn theradio-based position estimation or to calculate the radio-based secondposition information. As a result, learning by the second estimator 14can be performed on the basis of the result in a case where thecamera-based position estimation has high reliability, making itpossible to enhance the accuracy of estimating the second positioninformation by the second estimator 14.

Third Embodiment

While the second embodiment described above is a case where reliabilityis obtained on the basis of the information obtained from the imager 20,the present embodiment is intended to obtain the reliability further onthe basis of the information obtained from the communicator 30.

FIG. 7 is a block diagram illustrating a configuration of the positionestimator 10 according to the present embodiment. The difference fromthe second embodiment described above is that the radio receptioninformation in the communicator 30 is also input into the reliabilityacquirer 16.

The reliability acquirer 16 obtains the reliability in the estimation ofthe first position information on the basis of the radio receptioninformation from the communicator 30 together with the imaginginformation from the imager 20. The radio reception information receivedby the communicator 30 is used as supplementary information to grasp thepresence or absence of the target terminal 40 in the imaging informationor the number of target terminals 40.

The use of the radio enables acquisition and recognition of anidentifier (ID) of a radio signal. The ID of the transmission source ofthe radio signal is recognized via decoding processing of the receivedradio signal, making it possible to accurately grasp the number oftarget terminals 40 surrounding the communicator 30. Furthermore, byestimating at least one of the received power, the arrival time, thearrival direction, etc. of the signal transmitted from the targetterminal 40, it is possible to recognize the approximate position of thetarget terminal 40.

In the present embodiment, by inputting these pieces of informationobtained from the radio reception signal to the reliability acquirer 16,it is possible to more accurately calculate the reliability of theposition information estimated from the imaging information of theimager 20. Specifically, the number of the target terminals 40 and theposition-related information obtained from the imaging information ofthe camera are compared with corresponding information obtained from theradio reception information, and in a case where the correlation betweenthese is high, it is possible to calculate and obtain the reliability asa high level.

As described above, according to the present embodiment as well,similarly to the above-described embodiment, it is possible to enhancethe position estimation accuracy by performing camera-based and theradio-based estimation and possible to determine whether to learn theradio-based position estimation or to calculate the radio-based secondposition information on the basis of the reliability of the camera-basedposition estimation. Furthermore, the use of the radio receptioninformation enables calculation of the reliability more accurately,making it possible to enhance the accuracy of the learning processing ofthe radio parameters and the position estimation processing by theradio.

Fourth Embodiment

While each of the embodiments described above is a case where there isan imager on the position estimating apparatus 1 side, the presentinvention is not limited to this configuration. That is, the targetterminal 40 may include the imager, and the position may be estimatedusing the imaging information captured by the imager provided in thetarget terminal 40.

FIG. 8 is a diagram schematically illustrating the position estimatingapparatus 1 and the target terminal 40 according to the presentembodiment. As illustrated in FIG. 8, the target terminal 40 accordingto the present embodiment includes an imager 42. The target terminal 40obtains imaging information using its imager 42, transmits imaginginformation to the position estimator 10 via a communicator 44 toperform position estimation.

Operation of the position estimator 10 using the imaging information andthe radio reception information is similar to the operation in each ofthe above-described embodiments.

As described above, according to the present embodiment, it is possibleto perform position estimation similar to the estimation in each of theabove-described embodiments by a camera or the like included in thetarget terminal 40.

Modification of Fourth Embodiment

FIG. 9 is a diagram illustrating a configuration of the target terminal40 and the communicator 30 according to a modification of theabove-described fourth embodiment.

The target terminal 40 includes the imager 42, the communicator 44, anda position estimator 46.

The imager 42 corresponds to the imager 20 of the above-describedembodiment, the communicator 44 corresponds to a communicator 30, andthe position estimator 46 corresponds to the position estimator 10. Inthis manner, the target terminal 40 may include necessary functions.

The position estimator 46 of the target terminal 40 performs positionestimation of oneself (hereinafter referred to as self-positionestimation) using imaging information captured by the imager 42. In thismanner, the first position information of oneself is estimated from theimaging information captured by the imager 42.

The self-position estimation may be performed by calculating a distanceto a feature point included in imaging information by the imager 42 orby using a technique of simultaneous localization and mapping (SLAM) ofsimultaneously performing object recognition and environment mapcreation.

The position estimator 46 may obtain the reliability in estimating thefirst position information, that is, the reliability of self-positionestimation. Acquisition of the reliability of self-position estimationis performed on the basis of the number of feature points, thebrightness of the imaging environment, or the like.

The communicator 44 receives radio signals from the surroundingcommunicators 30A to 30D and performs position estimation processingusing the radio reception information. In a case where the positionestimator 46 determines that the reliability of the self-positionestimation in the target terminal 40 is high, the position estimator 46may use the self-position as the second position information based onthe self-position estimation result. Similarly to the above-describedembodiment, correction may be performed on the basis of the receptioninformation of radio waves transmitted from the communicator 30 toestimate the second position information. Note that the reliability neednot necessarily be determined, similarly to the above-describedembodiment. Moreover, the number of communicators 30 may be any number.

Furthermore, the position estimator 46 may learn the radio parameter onthe basis of the self-position estimation result. Then, in a case whereit is determined that the reliability is low, the second positioninformation based on the radio reception information may be estimatedusing the learned radio parameter to set the second position informationas the self-position. With this configuration, it is possible to performhighly accurate self-position estimation using both the camera-based andthe radio-based estimation, similarly to each of the above-describedembodiments.

As illustrated in FIG. 1, the estimated position information may be heldby the position estimator 10 outside the target terminal 40, or may beheld in the position estimator 46. Alternatively, the information may betransmitted to a server (not illustrated) or the like. With theabove-described configuration, even in a case where the positionestimator 10 is not provided outside, the communicator 30 is controlledto perform position estimation in the position estimator 46, enablingthe target terminal 40 including the camera and a radio function to knowits self-position accurately.

The components of the position estimating apparatus 1 of the presentembodiments may be implemented by dedicated hardware such as anintegrated circuit (IC) on which a processor or the like is mounted. Forexample, the position estimating apparatus 1 may include a receptioncircuit to implement the receiver in the communicator 30 or thecommunicator 44, a transmission circuit to implement the transmitter,and a control (processing) circuit as a controller to implement theposition estimator 10 or the position estimator 46. The internalconfiguration of the controller may also be implemented by a dedicatedcircuit. Alternatively, the components may be implemented using software(a program). In the case of using software (program), theabove-described embodiments can be implemented by using ageneral-purpose computer apparatus as basic hardware and causing aprocessor such as a central processing unit (CPU) mounted on thecomputer apparatus to execute the program. In a case where a portion ofthe function is configured by software, a program to implement at leasta portion of the function of the position estimating apparatus 1 may bestored in a recording medium such as a flexible disk and a CD-ROM to beloaded on a computer to be executed. The recording medium is not limitedto a detachable one such as a magnetic disk and an optical disk, and maybe a fixed type recording medium such as a hard disk apparatus and amemory.

Moreover, a program to implement at least a portion of the function ofthe position estimating apparatus 1 may be distributed via acommunication channel (including radio communication) such as theInternet. Furthermore, the program may be encrypted, modulated, orcompressed, and then distributed via a wired or radio channels includingthe Internet, or may be stored in a recording medium to be distributed.

The terms used in the present embodiment are to be interpreted broadly.For example, the term “processor” may include a general-purposeprocessor, a central processing unit (CPU), a microprocessor, a digitalsignal processor (DSP), a controller, a microcontroller, and a statemachine. Depending on the situation, the “processor” may refer to anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA), a programmable logic circuit (PLD), or the like. The“processor” may refer to a combination of processing apparatuses such asa plurality of microprocessors, a combination of a DSP and amicroprocessor, and one or more microprocessors cooperating with a DSPcore.

As another example, the term “memory” may include any electroniccomponent capable of storing electronic information. The “memory” can bea random access memory (RAM), read only memory (ROM), programmable readonly memory (PROM), erasable programmable read only memory (EPROM),electrically erasable PROM (EEPROM), nonvolatile random access memory(NVRAM), flash memory, magnetic or optical data storage, being readableby the processor. In a case where the processor executes both or one ofreading and writing of information onto the memory, this can be referredto as electric communication of the memory with the processor. Thememory may be integrated into the processor, and this case can also bereferred to as electric communication of the memory with the processor.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel embodiments described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the inventions.

The communicator 30 or the communicator 44, for example, obtains radioreception information by using radio waves, but it is not limitedthereto. In addition to the radio waves, radio reception information maybe obtained using other means such as acoustic waves, light waves toperform position estimation.

The invention claimed is:
 1. An electronic device for estimating aposition of a target apparatus external to the electronic device, theelectronic device comprising: receiver circuitry configured to receive aradio wave sent from the target apparatus, the radio wave includingradio information available for estimating the position of the targetapparatus; and controller circuitry configured to: determine firstposition information of the target apparatus using information obtainedfrom a captured image of the target apparatus; determine whether thefirst position information being obtained is appropriate positioninformation; estimate, when the first position information is determinedto be appropriate position information, second position information ofthe target apparatus based on at least one of (i) the first positioninformation or (ii) one or more radio characteristics of the radio wave;and estimate, when the first position information is determined not tobe appropriate position information, the second position information ofthe target apparatus based on one or more radio characteristics of theradio wave, wherein an accuracy of determining the first positioninformation when the first position information is determined to beappropriate position information is higher than an accuracy ofdetermining the first position information when the first positioninformation is determined not to be appropriate position information. 2.The electronic apparatus according to claim 1, wherein the controllercircuitry is configured to: learn a radio parameter related to the radioinformation to determine the second position information when the firstposition information is appropriate position information, the radioparameter being based on the first position information, and estimatethe second position information of the target apparatus based on atleast one of the learned radio parameter or one or more radiocharacteristics of the radio wave when the first position information isnot appropriate position information.
 3. An electronic device forestimating a position of a target apparatus external to the electronicdevice, the electronic device comprising: receiver circuitry configuredto receive a radio wave sent from the target apparatus, the radio waveincluding radio information available for estimating the position of thetarget apparatus; and controller circuitry configured to: determinefirst position information of the target apparatus using informationobtained from a captured image of the target apparatus; determinewhether the first position information being obtained is appropriateposition information; and estimate, when the first position informationis determined to be appropriate position information, second positioninformation of the target apparatus by correcting or arithmeticallyprocessing the first position information using one or more radiocharacteristics of the radio wave, wherein an accuracy of determiningthe first position information when the first position information isdetermined to be appropriate position information is higher than anaccuracy of determining the first position information when the firstposition information is determined not to be appropriate positioninformation.